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Abstract 

Over the past decade, great progress has been made in improving the quality and availability of Open 
Educational Resources (OER). OER proponents often discuss the ability for users to revise and remix OER 
to make them more suitable for local contexts; however, much OER goes unmodified. This note from the 
field examines the efforts of NetEase Online Open Courses, a Chinese organization, to take more than two 
thousand Khan Academy videos and translate them into Chinese. We provide background on this initiative 
and also analyze site metrics to determine what type of use these derivative OER have received. 
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Introduction 

An important topic in education concerns the use of Open Educational Resources (OER). D’Antoni (2009) 
cited the William and Flora Hewlett Foundation, a key donor for OER, as stating that OER are “resources 
that reside in the public domain or have been released under an intellectual property license that permits 
their free use or re-purposing by others” (p. 4). 

A key aspect of this definition is that OER permits free re-purposing; in other words, individuals can legally 
revise or remix OER. Hilton, Wiley, Stein, and Johnson (2010) examine several issues relating to OER remix 
and point out that there are multiple gradations in how “open” a resource really is. The resources that are 
most open allow for complicated revisions, such as translation. Although OER proponents often point to 
the legal permission to revise as a vital benefit of OER, an important question is whether people actually 
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take advantage of the legal permissions afforded by OER. A related issue surrounds the potential impact of 
revised OER. Do people use revised versions of OER? In other words, do revisions actually extend the 
influence of OER? How do users perceive revised OER? Compared with other aspects of OER research, 
these important questions have yet to be fully examined. The purpose of the present case study is to examine 
one revision of one set of OER (Khan Academy videos). First, we review the literature pertaining the revising 
OER, as well as Khan Academy. 


Review of Literature 


Revision of OER 

Duncan (2009) found that very few OER adopters revise or remix the work. Revision is defined as a 
modification to the OER, such as a translation, or an addition or deletion of material. When additional 
resources are combined with an OER, the new work is considered to be a remix. In Duncan’s study of an 
OER collection containing 3,519 utilized components, less than 3% were translated, and less than 3% were 
modified in some other way. Thus, in this study, less than 6% of the OER in the reviewed set were revised 
or remixed. 

Additionally, Petrides, Nguyen, Jimes, and Karaglani (2008) noted that in the same OER collection 
observed in Duncan’s analysis, 88% of those who altered the open resources were the original authors 
themselves. If a fundamental foundation of the OER model is the ability for others to modify the materials, 
then the findings of Petrides et al. (2008) convey a limited extension of a principle objective. Consequently, 
this report challenges the opinion that people are anxious to refashion resources for their own purposes. 

To further explore the customization of open resources, Hilton, Lutz, and Wiley (2012) investigated the 
degree to which instructors reused, revised, and remixed Flat World Knowledge (FWK) open textbooks. By 
comparing custom texts to the original material, the examination catalogued additions, deletions, reorders 
(changing the sequence of the material), and remixes (introducing the content of another FWK resource 
into the custom text). This study likewise observed low revision and remix rates, with only 7.5% of the texts 
customized over the two-year period. More significantly, a noteworthy pattern became apparent; the 
simpler to make the change, the more of those changes were observed. Deletions, the easiest alteration, 
were found in 60.32% of customized materials, followed by re-orderings in 40.49%. Additions were made 
in 16.60% of customized materials, and finally, remixes, the most challenging modification, occurred in 
only 1.62% of the remixes. 

Amiel (2013) examined decisions and challenges accompanying the creation of a work by translating and 
remixing open education resources. Amiel produced an instructional booklet on OER for Brazilian K-12 
teachers and chronicled the efforts that it took to create this resource. Amiel identified six different routes 
a utilized original resource could take. A contribution could be “in complete form attempting to maintain 
the original meaning and context (revision); adapting language to reflect local meaning and needs 
(revision/remix); modifying substantially to the point of departing from the original source (remix); only 
as a source of inspiration, demonstrating a need to create content based on the principles and ideas but not 
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borrowing directly from the source itself (reference); identifying instead other, more compatible resources 
(search); developing entirely new content when none was available (create)” (Amiel, 2013, pp. 133-134). 

When creating a new OER, Amiel selected resources with more open licenses over superior material with 
narrowed licensing. Amiel found several concerns with this English-based westernized OER that invoked 
multiple concerns, such as cultural barriers. Amiel concluded that although OER are available to be altered, 
in practice, performing the actual remix is not an easy task. 

While revising OER at the level of discrete learning resources appears to happen relatively irregularly, 
research indicates that some important books, or large-scale OER projects, do receive extensive revisions. 
For example, Hilton (2009) found that many people were willing to do large scale revisions (e.g., 
translation, creating an audio version, etc.) of key works such as Larry Lessig’s Free Culture. Carson, 
Kanchanaraksa, Gooding, Mulder, and Schuwer (2012) point out that in the case of MIT OpenCourseWare, 
extensive revisions in the form of translations have taken place, including multiple MIT open courses being 
translated into Chinese, Spanish, Portuguese, Farsi, and Thai. Lee, Lin, and Bonk (2007) chronicle the effort 
that was undertaken by over two thousand volunteers to produce the Chinese translation. Thus, while OER 
generally do not appear to be frequently revised, selected large-scale initiatives have been revised, with the 
revision frequently being translation. 

The focus of the present study is a translation of Khan Academy videos into Mandarin; we next discuss 
additional context for this study by first providing an overview of Khan Academy, as well as what the 
literature says about its impact on student learning. 

Khan Academy 

Initially founded as a hobby of a hedge fund analyst who was a long distance tutor for his cousins, Khan 
Academy now has impacted millions of lives across the globe (“Stories,” 2016). Salman Khan transitioned 
from tutoring using Yahoo’s Doodle notepad to creating YouTube videos, and after unexpected popularity, 
established Khan Academy in October 2006. Three years later, Khan abandoned his finance work to focus 
on his nonprofit educational organization full time—in a walk-in closet (Noer, 2012). 

Khan Academy has released more than 5,500 videos on YouTube and its website, khanacademv.org. in 
addition to hundreds of thousands of practice problems that students can work on at their own pace. 
Subjects covered in the micro-lectures include math from the elementary to college level, science and 
engineering, computing, arts and humanities, economics and finance, and test prep for exams such as the 
SAT, MCAT, and GMAT (“About,” 2016). 

Murphy, Gallagher, Krumm, Mislevy, and Hafter (2014) examined the efficacy of Khan Academy as a 
supplemental resource in schools between 2011 and 2013. Two thousand students in grades 5-10 
participated in the study; nine sites, 20 schools, and 70 teachers were involved. Findings from the study 
include the following: student perceptions of their time spent on Khan Academy was highly positive, student 
engagement was generally high, and a majority of teachers stated that they planned to continue using the 
Academy after the conclusion of study. Teachers also reported that the program increased their capacity to 
support students across multiple areas, and that they found Khan Academy reports useful to review student 


307 



Khan Academy Videos in Chinese: A Case Study in OER Revision 
Rao, Hilton, and Harper 


progress. Murphy et al. (2015) found a positive correlation between increased Khan Academy use and “(1) 
improvements in student test scores, and (2) improvements in three of the four self- reported 
nonachievement outcomes - math anxiety, math self-concept, and academic efficacy (i.e., belief in one’s 
ability to succeed in academic endeavors)” (p. 12). 

In a separate research study, Cohen and Phillips (2015) examined an implementation of Khan Academy 
across Idaho 47 different schools in 33 that focused on personalized learning. Data from 5,309 students 
between 3 rd and 8 th grade showed positive correlations between the usage of Khan Academy and academic 
progress (Cohen & Phillips, 2015). In a comparison between Khan Academy mission (a personalized set of 
grade-level content) completion and target growth, “students who completed 0-10% of their mission on 
average grew as expected,” “students who completed 40% or more of their mission on average grew 1.5 
times more than their expected growth in one year,” and “students who completed 60% or more of their 
mission on average grew 1.8 times their expected growth in one year” (Cohen & Phillips, 2015, p. 22). Other 
studies have found learning gains when Khan Academy videos were implemented in classroom settings 
(Castillo & McIntosh, 2012; Park, 2016). 

Khan Academy licenses its videos with a Creative Commons BY-NC-SA license. This open license allows 
people to retain, reuse, revise, remix, and redistribute the videos. Thus, Khan Academy legally grants 
permission for people to edit their videos and share them in different formats, as long as it is done for non¬ 
commercial purposes. 

Khan Academy Translations 

Because of the value of Khan Academy videos, significant efforts have been made to translate them into 
other languages. Khan Academy has a volunteer translation portal, by which much of its content is available, 
following a volunteer translation model that has been used by others (Beaven, Comas-Quinn, Hauck, de los 
Arcos, & Lewis, 2012). However, in the case of the Chinese translation of Khan Academy videos, the 
translation was done by NetEase Open Courses, a subsidiary of called NetEase (M Hi). NetEase is an 
Internet company based in Beijing, China that focuses a variety of Internet services including email and 
online gaming. As part of a non-profit initiative within the company, NetEase Open Courses hosts multiple 
OER and has been involved in a variety of initiatives relating to making educational content freely available, 
such as working with Coursera to deliver free online courses (Penna, 2013). 

In 2012, Khan Academy announced, 

At present, 927 videos from Khan Academy have been translated by NetEase Open Courses, covering 
mathematics, chemistry, biology, geometry, physics, and other disciplines...NetEase is the first portal 
site in China to partner with Khan Academy to localize its content and this greatly enhances NetEase 
Open Courses’ content and user base. Meanwhile, the cooperation with NetEase will promote the 
localization of Khan Academy courses in China, to achieve the purpose of providing a world class 
education for anyone anywhere (“NetEase subtitles,” 2012). 

As of September, 2016, NetEase has translated 2,931 Khan Academy videos into Chinese, across 88 different 
courses (e.g., U.S. History, see appendix for complete course list). 
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In the present study we examine the impact of NetEase’s revision of OER, specifically the Khan Academy 
videos, by asking the following questions: 

1. How frequently have the Chinese Khan Academy videos been viewed? 

2. Which Chinese Khan Academy videos are most popular? 


Method 

We used analytics data provided by NetEase Open Courses to determine which videos had been viewed 
most frequently. The two principal ways that people can access the Khan Academy videos in Chinese are 
through NetEase’s website ( http://open.16.com/khan/ ) and their apps. Unfortunately, accurate analytics 
data were not available prior to 2016; thus, we used the analytics data available between January 1 - 
September 30, 2016 to determine the number of page views. These data were provided by NetEase Open 
Courses and capture visits that took place through the NetEase Open Courses app, as well as their web 
portal. 


Results 


Total Number of Views 

Table 1 lists the top 10 most frequently viewed courses between January 1 - September 30, 2016, for both 
the App and website views. 

Table 1 

Top Viewed Khan Academy Courses, January 1 - September 30, 2016 


Course 

App view 

Website views 

Total views 

Finance 

15,185,984 

1,419,990 

16,605,974 

Microeconomics 

6,921,772 

921,162 

7,842,934 

M acroeconomics 

6,047,580 

803,000 

6,850,580 

Linear Algebra 

5,883,658 

928,620 

6,812,278 

Statistics 

4,167,504 

1,020,360 

5,187,864 

Chemistry 

3,618,912 

386,100 

4,005,012 

Physics 

3 , 443,462 

422,444 

3,865,906 

Probability 

3,429,712 

347,886 

3 , 777,598 

Health and Medicine 

3,164,194 

305,646 

3,469,840 

Differential Calculus 

2,675,266 

347,842 

3,023,108 
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6,903,050 


61,441,094 


Total 


54,538,044 


Finance, Economics, Math, and Science are the most popular topics for the Chinese Khan Academy videos. 
Access to videos is much more likely to come from the NetEase app than the website. Most importantly, it 
is clear that there is a large amount of interest in these videos, with the top 10 courses garnering more than 
60 million views. 


Discussion 

These data prompt several observations about the ability to revise OER and its attendant usefulness. First, 
while studies have shown that most OER are not revised, this study clearly demonstrates the deep impact 
that revision can have. In this study, we found that revised Khan Academy videos received over 60 million 
page views between January and September of 2016. Given that many of these videos have been available 
for four years, this potentially puts the total page views at over 200 million. Moreover, the impact of the 
Chinese-translated Khan Academy videos is further understated by the fact that we have only included data 
based on visits to NetEase Open Courses. Because these videos are openly licensed, many enterprising users 
have posted the NetEase Open Courses videos on a variety of websites, such as Youku.com. 

Thus, while OER generally do not appear to be revised, in some instances the open licenses that undergird 
OER allow for deep and significant expansions (e.g., Carson et al., 2012) The present study falls into this 
category given the large number of views that the Chinese Khan Academy videos have received. This result 
underscores the benefits of the “Open” in OER, as opposed to resources that are simply “free.” 


Conclusion 

This brief note from the field demonstrates the clear value that revisions can have in extending the influence 
of OER. Khan Academy videos are evidently a valuable resource and their educational value has been 
extended as a result of being OER. Without the open license that allowed for the remix of these videos, the 
viewership would be significantly less. While most OER do not receive dramatic revisions, this study 
illustrates the value of open licenses and the additional learning that can take place when resources are 
given open, rather than restricted licenses. 
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Appendix 

Khan Academy Courses Translated into Chinese by NetEase 

1. Finance 

2. Microeconomics 

3. Macroeconomics 

4. Statistics 

5. linear Algebra 

6. Basic Chemistry 

7. Physics 

8. Probability 

9. Health care and medicine - Circulatory System 

10. Calculus 

11. Brain Teasers 

12. Basic Algebra 

13. Precalculus 

14. Introduction to art history 

15. Triangle 

16. Computer Science 

17. United States History 

18. Modern Cryptography 

19. Brain Teasers (Chinese Voiceover) 

20. Doodling in math 

21. Complex Number 

22. Finance (Chinese Voiceover) 

23. Logarithms 

24. Cosmology and Astronomy 

25. Ratios & proportions 

26. Differential Equations 

27. Organic Chemistry 

28. European History 

29. Banking and Money 

30. Venture Capital and Capital Market 

31. Graphing linear equations and inequalities 

32. United States History(Chinese Voiceover) 

33. Algebra Examples 
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34. Euclidean Geometry 

35. Valuation and Investing 

36. 400 C.E. Ancient Cultures 

37. Singapore Math 

38. Information Theory 

39. Current Economics 

40. Ancient Cryptography 

41. Arithmetic and Pre-Algebra: Multiplication and division 

42. Differential Equations 

43. Basic geometry 

44. Absolute Value 

45. Venture Capital and Capital Market 

46. Logarithms 

47. Graphing linear equations and inequalities 

48. Arithmetic and Pre-Algebra: Addition and subtraction 

49. Differential Equations 

50. Computational Number Theory Prime Adventure 

51. Arithmetic and Pre-Algebra: Fractions 

52. Geometry: Angles 

53. Information Theory 

54. Polygon, Quadrangle Parallelogram 

55. Arithmetic and Pre-Algebra: Number properties 

56. Health care and medicine: Renal system 

57. Ancient and Medieval Histoiy 

58. Logarithms 

59. Modern Cryptography 

60. California Standards Test: Algebra I 

61. Arithmetic and Pre-Algebra: Exponents (basic) 

62. GMAT: Problem Solving (Math) 

63. Arithmetic and Pre-Algebra: Negative numbers 

64. Arithmetic and Pre-Algebra: Fractions 

65. Arithmetic and Pre-Algebra: Decimals 

66. Absolute value 

67. Indian Institute of Technology joint entrance examination 

68. CAHSEE Example Problems 

69. Arithmetic and Pre-Algebra: Number properties 

70. Arithmetic and Pre-Algebra: Ratios and proportions 
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71. Geometry work examples 

72. Ancient and Medieval History 

73. SAT Preparation (Math) 

74. California Standards Test : Geometry 

75. Information Theory 

76. Computational Number Theory Prime Adventure 

77. Absolute value 

78. Polygon, quadrangle, parallelogram 

79. Geometry: Angles 

80. Algebra: ck12.org Algebra 1 Examples 

81. GMAT: Problem Solving (Math) 

82. Arithmetic and Pre-Algebra: Factors and multiples 

83. Arithmetic and Pre-Algebra: Number properties 

84. Arithmetic and Pre-Algebra: Exponents (basic); 

85. GMAT: Data Sufficiency 

86. Computational Number Theory Prime Adventure 

87. Arithmetic and Pre-Algebra: Order of operations 

88. California Standards Test: Geometry 
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